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Abstract 


2 Characterization of the error associated to satellite rainfall estimates is a necessary 

3 component of detenninistic and probabilistic frameworks involving spacebome passive and 

4 active microwave measurements for applications ranging from water budget studies to 

5 forecasting natural hazards related to extreme rainfall events. We focus here on the error 

6 structure of Tropical Rainfall Measurement Mission (TRMM) Precipitation Radar (PR) 

7 quantitative precipitation estimation (QPE) at ground. The problem was addressed in a 

8 previous paper by comparison of 2A25 version 6 (V6) product with reference values derived 

9 from NOAA/NSSL’s ground radar-based National Mosaic and QPE system (NMQ/Q2). The 

10 primary contribution of this study is to compare the new 2A25 version 7 (V7) products that 

1 1 were recently released as a replacement of V6. This new version is considered superior over 

12 land areas. Several aspects of the two versions are compared and quantified including 

13 rainfall rate distributions, systematic biases, and random errors. All analyses indicate V7 is 

14 an improvement over V6. 

15 

16 Key words: satellite-based rain estimation, radar, QPE, conditional bias, random error 
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1 . 


Introduction 


Given their quasi-global coverage, satellite-based quantitative rainfall estimates are 
becoming widely used for hydrologic and climatic applications. Characterizing the error 
structure of satellite rainfall products is recognized as a major issue for the usefulness of the 
estimates (Yang et al. 2006; Zeweldi and Gebremichael 2009; Sapiano and Arkin 2009; 
Wolff and Fisher 2009) as underlined by the Program to Evaluate High Resolution 
Precipitation Products (Turk et al. 2008) led by the International Precipitation Working 
Group (IPWG; see http://www.isac.cm~.it/~ipwg/) . In this study, we focus on the TRMM 
Precipitation Radar (PR) quantitative precipitation estimation (QPE) product. The TRMM- 
PR is currently the only active instrument dedicated to the measurement of rainfall from a 
satellite platform conjointly with a radiometer (TMI). PR measurements are considered as 
the starting point for subsequent algorithms that use microwave measurements from low- 
earth orbiting satellites and for combined end products that utilize data from geostationary 
satellites (e.g., Yang et al. 2006; Wolff and Fisher 2008, Ebert et al. 2007, Berges et al. 
2010, Ushio et al. 2006). Our aim is to compare the new PR 2A25 version 7 (V7) products 
that were recently released as a replacement for version 6 (V6). This new version is 
considered superior over land areas compared to the previous versions due to changes to the 
vertical profile of hydrometeor characteristics, which impacts the reflectivity-to-rainfall rate 
(Z-R) relationship and attenuation correction. Finally, a correction for non-uniform beam 
filling (NUBF) effects was reintroduced. 

The methodology and framework followed here are described in a previous paper 
dedicated to the evaluation of 2A25 V6 (Kirstetter et al. 2012). The PR QPE product was 
assessed with respect to an independent reference rainfall data set derived from high- 
resolution measurements using NOAA/NSSL’s ground radar-based National Mosaic and 
QPE system (NMQ/Q2; Zhang et al. 2011a). These products yield instantaneous rainfall rate 
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products over vast regions including the conterminous US (CONUS). A systematic and 
comprehensive evaluation for regions over the southern CONUS was perfonned by 
characterizing errors in PR estimates by matching quasi-instantaneous data from Q2 at the 
~5-km pixel measurement scale of PR in order to minimize uncertainties caused by 
resampling. The study used three months (March-May 2011) of satellite overpasses over the 
lower CONUS. Despite the seemingly short period for evaluation, the use of gridded Q2 data 
for reference provided a large sample size totalling 392 713 comparisons. The exact same 
reference dataset that was used to evaluate V6 is used in this study for V7. 

The PR and Q2 reference data are briefly described in section 2. In section 3 we assess 
the differences in the probability density functions (PDFs) of rain rate for 2A25 V6 and V7 
and their ability to represent rainfall variability. A quantitative comparison of empirical error 
models for V6 and V7 estimates versus reference rainfall is provided in section 4. The paper 
is closed with concluding remarks in section 5. 

2. Data sources 

a) Q2-based reference rainfall 

All significant rain fields observed coincidentally by TRMM overpasses and the 
NEXRAD radar network from March to May 2011 are collected. The Q2 products closest in 
time to the TRMM satellite local overpass schedule time are used. The NOAA/NSSL 
National Mosaic and Quantitative Precipitation Estimation system (NMQ/Q2) 
(http://nmq.ou.edu; Zhang et al. 2011a) is a set of experimental radar-based products 
comprising high-resolution (0.01°, 5 min) instantaneous rainfall rate mosaics available over 
the CONUS (Zhang et al. 2005; Lakshmanan et al. 2007; Vasiloff et al. 2007; Kitzmiller et 
al. 2010). One should note that it is not possible to “validate” the PR estimates in a strict 
sense because independent rainfall estimates with no uncertainty do not exist. Many errors 
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1 affect the estimation of rainfall from ground-based radars, such as non-weather echoes, 

2 NUBF, range-dependency due to Vertical Profile of Reflectivity (VPR) variability, 

3 conversion of Z-to-R, and calibration of the radar signal. While several procedures are 

4 already in place within the Q2 system to correct for these errors, the following post- 

5 processing steps were taken to refine the reference data set as much as possible: (i) adjusting 

6 instantaneous Q2 products using co-located rain gauge observations (corrects for inaccurate 

7 Z-R relationship and calibration errors) and (ii) filtering data through a Radar Quality Index 

8 (RQI, Zhang et al. 2011b) (eliminates overestimation in the bright band and mitigates range 

9 dependency caused by VPR effects). One must keep in mind these improvements may not 

10 screen out all possible errors in ground-based radar estimates. The reference rainfall R is 

11 a Q2 rainfall mean computed to match each PR pixel by considering the power density 

12 function of the PR beam. A standard error is computed alongside the mean reference rainfall 

13 value: (T footpnnt , which represents the variability of the Q2 rainfall (at its native resolution) 

14 inside the PR footprint and is used to select the PR-Q2 reference pairs for which the R ref is 

15 trustworthy (see Kirstetter et al. 2012 for more details). The reference pixels are segregated 

16 into “robust” (R t > n ) and “non robust” (R t < n ) estimators. Non-robust 

v ref footprint ' v ref footprint ' 

17 reference values are discarded for quantitative comparison. The PR rainfall statistical 

18 characteristics are preserved because the product remains free of undesirable impacts caused 

19 by resampling. 

20 

21 b) Precipitation Radar (PR) based rainfall 

22 The PR measures reflectivity profiles at Ku band. Surface rain rates are estimated over 

23 the southern US up to a latitude of 37°N (see Fig. 1, Kirstetter et al. 2012). The scan 

24 geometry and sampling rate of the PR lead to footprints spaced approximately 5.1 km in the 
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horizontal and along-track, over a 245-km-wide swath. The TRMM product used in this 

2 work is the PR 2A25 product (versions 6 and 7) described in Iguchi et al. (2000, 2009). The 

3 2A25 algorithm relies on a hybrid attenuation correction method that combines the surface 

4 reference technique and Hitschfeld-Bordan method (Iguchi et al. 2000; Meneghini et al. 

5 2000, 2004). Retrieval errors of the algorithm have mainly been attributed to the uncertainty 

6 of the assumed drop size distribution (DSD), incorrect physical assumptions (freezing level 

7 height, hydrometeor temperatures) and NUBF effects (Iguchi et al. 2009). Some of the 

8 weaknesses in performance with V6 (i.e., underestimation of rain-rates) over land compared 

9 to over sea previously reported (Wolff and Fisher 2008; Iguchi et al. 2009) are expected to 

10 improve as Z-R relationships over land were recalibrated and the NUBF correction, which 

1 1 was abandoned in V6, was re-introduced in the new V7 product. 

12 

13 3. Rainfall data analysis 

14 a) Probability distributions by occurrence and by rain volume 

15 Hereafter, the PR rain estimates are the conditional ones (non-zero rainfall) coincident 

16 and collocated with non-zero Q2 reference estimates. In addition, the “robust” (R- ref > 

17 n ) rain rates dataset is used as reference. Two PDFs for PR versus 02 reference 

footprint y ^ 

18 rainfall are computed and shown in Fig. 1: (i) the PDF by occurrence (PDF C ) and (ii) the 

19 PDF by rain volume (PDF V ) (Wolff and Fisher 2009; Amitai et al. 2009, 2011; Kirstetter et 

20 al. 2012). The PDF C provides statistical information on the rain rate distribution and 

21 highlights the estimates’ sensitivity as a function of rain rate. The PDF V represents the 

22 relative contribution of each rain rate bin to the total rainfall volume. 

23 Compared to Q2’s reference PDF C , both 2A25 versions tend to overestimate light rain 

24 rates ( — [0.3-0.5] mm h' 1 ) and demonstrate poor detection of the lightest rain rates (below 

25 ~0.3 mm h" 1 ). A possible explanation is the edges of rain areas might be only partially 
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1 detected by PR because they are associated with low rain rates and intermittency (Kirstetter 

2 et al. 2012). The detectability issue is related to the sensitivity of PR and is thus not readily 

3 correctable with an update to the processing algorithm. However, it is noted that the mode of 

4 V7’s PDF C is shifted towards higher values than V6’s and is more consistent with the mode 

5 of the reference PDF C . In examining the rain rate distributions by volume, we see the modes 

6 of PDF V for both V6 and V7 are shifted toward lower rain rates compared to the reference’s 

7 mode (~60 mm h" 1 ), which agrees with the results found in Amitai et al. (2006, 2009). This 

8 has been attributed to high rainfall rates (> 10 mm h" 1 ), which are likely underestimated by 

9 PR due to one or more of the following reasons: insufficient correction due to attenuation 

10 losses, NUBF effects, and inaccurate conversion from Z-to-R (Wolff and Fisher 2008). V7 

1 1 presents a PDF V in better agreement with the reference than V6. The mode of the PDF V has 

12 increased from 18 to 25 mm h , indicating a positive impact from the NUBF correction 

13 and/or Z-R improvements over land. 

14 

15 b) Correlations and biases 

16 Density-colored scatterplots of PR versus reference rainfall are presented for the two 

17 versions of 2A25 in Fig. 2. Improvements (i.e., increases) in V7 are evident particularly for 

18 reference rainfall values > 30 mm h' 1 . In addition, the underestimation from V6 at lighter 

19 rain rates (< 1 nun h' 1 ) has now been mitigated in V7. We also provide common comparison 

20 metrics in Table 1. A rainy pixel is included in the statistics if both PR and the reference are 

21 non-zero. The V6 and V7 estimates are both subjected to the same discrepancies in 

22 spatiotemporal matching with the Q2 reference, which is a source for differences on a point- 

23 to-point comparison basis, so their relative differences can be directly attributed to 

24 algorithms themselves. PR underestimates the mean reference rainfall values in both 

25 versions. However, the V7 products are less biased (-18%) than the prior version (-23%), 
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1 showing a positive impact of the new processing (i.e., recalibrated Z-R relationship over 

2 land and NUBF correction). The correlation coefficients between both versions of PR 

3 rainfall and Q2 reference estimates are moderate, but we note the correlation with V7 has 

4 improved. Increasing both the bias and the correlation of the 2A25 products is a significant 

5 achievement. In fact, it is generally recognized that it is difficult to improve one of these 

6 statistics without the expense of the other (Ciach et al. 2000). The correction of the largely 

7 underestimated rain rates in going from V6 to V7 (see Fig. 2) certainly contributes to this 

8 improvement. 

9 

10 c) Error models 

1 1 The uncertainties associated with satellite estimates of rainfall include systematic errors 

12 as well as random effects from several sources (Yang et al. 2006; Kirstetter et al. 2011). In a 

13 similar manner with Kirstetter et al. (2012), the departures of PR estimates from the Q2 

14 reference values are analyzed in this section on a point-to-point basis. With the true rainfall 

15 being unknown, the residuals are defined as the difference between the reference rainfall 

16 (R ) and the satellite estimates (R):s = (R - R ) . Only pairs for which R and R are 

17 both nonzero are considered in the calculations. The sets of s distributions are studied using 

18 the generalized additive models for location, scale, and shape (GAMLSS) technique (Rigby 

19 and Stasinopoulos 2001, 2005; Akantziliotou et al. 2002; Stasinopoulos and Rigby 2007). 

20 R ref is considered as the main driving (explanatory) variable conditioning the departures of 

21 PR estimates from reference values and we use the reverse Gumbel distribution 


22 




) to model the conditional residual distributions, 


23 where the location u (mean of the residual population) is to be linked to systematic errors 


24 and a (the standard deviation) is representative of random errors. 
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2 For a given conditional distribution of the response variable s ? the conditional quantiles 

3 can be expressed as a function of R ref . Figure 3 shows the residuals as a function of R ref as 

4 well as the fitted GAMLSS model for the two 2A25 versions. The conditional PDFs of 

5 residuals s present a high conditional shift from the 0 line and a high conditional spread. 

6 Note that for R cf > ~50mm h , the model is quite undetermined because of the lack of 

7 observed residuals. Both 2A25 versions present a tendency to underestimate high rain rates 

8 (negative median of residuals); V6 underestimates R ref = 20 mm h" 1 with an occurrence of 

9 80% and with a representative bias of -7 mm h" 1 while V7 underestimates the same reference 

10 value with an occurrence of 75% and with a representative bias of -6 mm h' 1 . There is an 

1 1 improvement with V7, but the remaining bias is likely to be due to an inaccurate Z-R 

12 relationship, NUBF effects and/or insufficient correction of PR attenuation losses at heavier 

13 rain rates. 

14 We consider the conditional median of the residuals to compare the systematic error 

15 component for V6 and V7 as well as the interquantile (90%- 1 0%) value to assess the random 

16 part of the error. Figure 4 shows the conditional biases and random errors of both versions of 

17 2A25 relative to the Q2 reference dataset. The underestimation with V6 and V7 over a large 

18 range of rain rates induces a global negative bias, which was evident in Table 1. The 

19 conditional biases of both versions relative to the reference are quite similar but with a slight 

20 improvement in V7. The random error increases consistently with R ref for both products. 

21 The random part of error for V7 is greater than V6, suggesting that other factors in addition 

22 to R ref could be considered to properly model the random error of V7 rain rate estimates. 

23 

24 4. Conclusions 
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1 A three-month dataset of gauge-adjusted, quality-filtered surface rainfall estimates from 

2 the NEXRAD-based Q2 has been used to compare and contrast PR-based 2A25 rainfall 

3 estimates from the older V6 algorithm and the newly released version (V7). V7 includes 

4 improvements in attenuation correction of the radar signal, a recalibrated Z-R equation for 

5 use over land areas, and a correction for NUBF effects was reintroduced. The comparisons 

6 have been performed at the PR-pixel resolution over the lower CONUS using a framework 

7 proposed in Kirstetter et al. (2012). Our analyses indicate that the bias of the rain rate 

8 estimates from V7 has been improved from a prior underestimation bias of -23% (from V6) 

9 to -18%. Moreover, this improvement in reducing bias is accompanied by an increase in the 

10 correlation coefficient from a prior value of 0.64 to 0.68; simultaneous improvement in both 

1 1 error metrics is quite challenging and was found to be a result of simultaneously correcting 

12 overestimation at lighter rain rates (< 10 mm hr" 1 ) and underestimation at high rain rates (> 

13 30 mm hr" 1 ). The former correction is most likely a result of the recalibration of the Z-R 

14 equation over land while the latter is likely a result from the NUBF correction; NUBF is 

15 known to cause underestimation at high rain rates (Iguchi et al. 2009). 

16 A statistical error model was developed for both versions of PR algorithms to separate 

17 conditional biases and random errors as a function of reference rainfall rate. The PR 

18 residuals are confirmed to be quite large even with the newer V7 due to the aforementioned 

19 combination of error factors. Presently, the error model only considers rainfall rate of the 

20 reference as the dominant factor. A more robust error model will include the primary, 

21 identifiable error sources in PR rainfall estimates. Future work will evaluate and quantify the 

22 relative contributions of PR rainfall estimation errors linked to additional factors such as 

23 rainfall type, off-nadir angle, NUBF, attenuation, as well as influence of the underlying 

24 terrain. 

25 

26 
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2 Table captions 

3 

4 Table 1. Performance criteria values for PR estimates: mean, standard deviation, mean 

5 relative error (MRE) and correlation (R) with respect to references. Only the reliable Q2 data 

6 are kept (see section 2.b) for references. 
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1 


2 Figure captions 

3 

4 Figure 1: Probability distributions of rain rates for the reference rainfall (grey) and for PR 

5 rainfall V6 (left) and V7 (right). The “robust” reference rain rates are used. The solid and 

6 dashed-dotted lines represent the distribution by volume PDF V and the distribution by 

7 occurrence PDF C respectively, while the grey and black lines represent the distributions for 

8 references and PR respectively. Note that the x-axis is in log-scale. 

9 Figure 2: Scatterplots of 2A25-V6 (left) and 2A25-V7 (right) versus reference rainfall 

10 (mm.h '). The first bisectors (solid lines) are displayed. 

1 1 Figure 3: PR residuals represented versus reference (top) and the corresponding GAM model 

12 fitted and represented by [5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95] conditional quantile lines 

13 (bottom) for 2A25-V6 (left) and 2A25-V7 (right). The dotted lines represent the cumulative 

14 distribution function of the reference rainfall. 

15 Figure 4: Conditional bias (median) of residuals (left) and conditional random error 

16 (interquantile 90%-10%) of residuals (right) for 2A25-V6 (blue) and 2A25-V7 (red) as a 

17 function of reference rainfall. 

18 

19 

20 
21 
22 
23 


18 



1 


2 

3 

4 TABLE 1. Performance criteria values for PR estimates: mean, standard deviation, mean 

5 relative error (MRE) and correlation (R) with respect to references. Only the reliable Q2 data 

6 are kept (see section 2.b) for references. 


PR -2A25 

Reference 

Version 6 

Version 7 

Mean 

7.27 

5.60 

5.97 

standard deviation 

13.76 

8.26 

9.8 

MRE / reference (%) 

- 

-23 % 

-18 % 

Correlation / reference 

- 

0.64 

0.68 
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Figure 1: Probability distributions of rain rates for the reference rainfall (grey) and for PR 
rainfall V6 (left) and V7 (right). The “robust” reference rain rates are used. The solid and 
dashed-dotted lines represent the distribution by volume PDF V and the distribution by 
occurrence PDF C respectively, while the grey and black lines represent the distributions for 
references and PR respectively. Note that the x-axis is in log-scale. 
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Figure 2: Scatterplots of 2A25-V6 (left) and 2A25-V7 (right) versus reference rainfall 
(min. IT 1 ). The first bisectors (solid lines) are displayed. 
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Figure 3: PR residuals represented versus reference (top) and the corresponding GAM 
model fitted and represented by [5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95] conditional 
quantile lines (bottom) for 2A25-V6 (left) and 2A25-V7 (right). The dotted lines represent 
the cumulative distribution function of the reference rainfall. 
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Figure 4: Conditional bias (median) of residuals (left) and conditional random error 
(interquantile 90%-10%) of residuals (right) for 2A25-V6 (blue) and 2A25-V7 (red) as 
a function of reference rainfall. 
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